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EfiBcient multiinaiking 



The present invention genecally xelates to the field of watennarking of media 
signals and more particidady to methods^ devices and a media signal for simplifying 
embedding of watermarks in different copies of a media signal 



It is widely known to watermark media signals in order to prevent illegal 
cqpying and distribution of the media signals. In tibis way a rightful owner of a media signal 
can detect if for instance copies of a media signal have been illegally made. 

With &e introduction of Interne^ there has been revolution within the jQled of 

10 distribution of media content, through allowing downloading of media signals fiom 
Electronic Content Delivery Systems. A content provider can then have a database of 
dififerent media signals and deliver copies of these to diEferent users via the Internet. A. 
purchaser of fliis content then often expects an instantaneous or direct delivery of access to 
the content In ocderto safeguard illegal copyingof these media signals it is ftien necessary to 

IS watermaik each copy distributed Willi a unique watermadc 

Watermarking is however in many cases relatively time consuming and 
involving complex computing opemtions. This means that if a content provider is to deliver 
many copies of a media signal at the same time to diEferent clients, the watermarking proems 
for each copy of the media signal will take a long time, which can lead to many clients 

20 receiving tibieir ordered media signal after a considerable delay, which is in many cases not 
acceptable. 

There is therefore a need for providing watermarking of different copies of a 
media signallhat is fiister, while still guaranteeing unique watemoarks for each watermarked 
copy. 

25 The document ^'Real-time Concepts for Block-based Watemaaxking Schemes'* 

by Michael Arnold and Oliver Lobisch, WEDELMUSIC Conference, 2002, Darmstadt, 
Geanany^ pp. 156 — 160, describes one system for reducing the computational time used. 
Here a media signal is watermarked with two different watermarks. The two watermarked 
copies of the media signal are then stored. Just prior to distributing the signal to different 
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clients, the watennarked signals are then noixed in a client dependent way in order to provide 
unique watermarks for different clients. This technique works best when the media signal is 
provided in frames. Then at every frame boundary one out of the two watermarked signals 
are mbced into the output signal. In this way an output media signal can be provided such that 

5 a sequence of the two watermarks are provided on a frame-per-frame basis. The specific 
sequence then represents the watermark of a certain client The document furthermore 
describes the use of a secret 1»/ timt detennines the order in which th^ 
provided. A similar system is also described in the document WO00/S60S9. 

Many types of watermaiking do not use fiames. In for instance mask coding 

10 there is a continuous sequence of symbols. Mixing of watermarks is possible in this 

environment The mixing is however difScult to perform and such ntiixed watermarks would 
fur&ermore be difficult to detect The payload edacity is furthermore limited by the 
maximum number of possible changes in the watermarking sequence. If the watermarking 
algorithm properties do not allow a relatively &st change in watermade and/or content size is 

IS relatively small, the total payload may be limited. The storage capacity for storing the 
watermarks furthermore needs to be increased significantly because of the storing space 
requirements for the two watermarks. 

There is thus a need for an alternative way of providing wateriiiarking of 
different copies of a media signal that is foster, while still guaranteeing unique watermarks 

20 for each watermarked copy and that furthermore limits foe storage capacity needed, allows a 
bigger variation of unique watermarks, espedaDy for small sized media content, and that can 
be used for also other watermarking techniques foat do not use fiames. 



25 It is thus an object of the present invention to provide watermarking of 

different copies of a media signal that is fast, while still guaranteeing unique watermarks for 
each watermarked copy, limits the storage capacity needed, allows a bigger variation of 
unique watermarks and that can be used for watermarking techniques that do not use frames. 

According to a first aspect of the present invention, this object is achieved by a 
30 method of simplifying embedding of watermarks in different copies of a media signal 
comprising foe steps of: 

determining watermarking properties dependent on a media signal, and 
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storing fhe signal dependent properties, such that the signal dependent 
properties can be used when embedding unique watermarks in different copies of the media 
signal. 

According to a second aspect of the present invention, this object is also 
achieved by a method of embedding a watermark in a media signal comprising the steps o£ 

receiving a media signal tngeHier with certain wfttermflrlring pr^yp^^ 

dependent on the media signal, and 

embedding a watermark based on the signal dependent properties in a copy of 
the media signaL 

According to a third aspect of the present invention, this object is furthermore 
achieved by a device for simplifying the embedding of watermarks m different copies of a 
media signal comprising a server unit including: 

a properties determining unit for determining signal dependent watermarking 
properties of a media signal , and 

a signal properties store for storing the signal dependent properties, such fliat 
the signal dependent prq[>erties can be used for embedding unique watermarks indifferent 
copies of the media signal. 

According to a fourth aspect of tiie present invention, this object is 
forthermore achieved by a device for embedding a watermark in a media signal comprising: 

a receiving unit for receiving a media signal together with certain 
watermaridng properties dependent on the media signal, and 

a watermarking unit arranged to embed a watermark based on the signal 
properties in a copy of the media signal. 

According to a fiflh aspect of the present invention, this object is also achieved 
by a signal for providing media content to a recipient comprising a media signal together 
with certain watermarking properties, which are dependent on the media signal. 

Claims 2 and 12 are directed towards sending the media signal together with 
information that is at least based on the signal dependent properties to a recipient 

CIaims3andl3 are directed towards embedding watennarks in copies of the 
media signal using the signal dependent p ro p erties and then sending the media signal 
including a unique watermark to recipients. In this way watennarks are embedded before 
delivery to recipients. 

Claims 4 and 1 4 are directed towards mixing watermarks in copies of the 
media signal. This enables the provision of unique watermarks in several copies of a media 
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signal based on a limited number of original watemiarks with only limited additional 
processing. 

Claims S and IS are directed towards sending the media signal and the signal 
dependent properties to a recipient In this way a trusted client can himself embed a unique 
5 watermark in the media signal. This also allows inspection of effects on the watermarking on 
tiie media signal beforehand, which allows modification of tiie signal dependent properties of 
a watermark before embedding. 

Claims 6, 10, 16 and 19 are directed towards lossless encoding and decoding 
of the signal dependent properties in tiie media signal This has the advantage of not requiring 
10 any extra bandwidth for the transmission of tibe signal dependent properties. 

Claims 7 and 17 are directed towards using a perceptual model of ahuman 
sensing system for determining the signal dependent properties. 

The present invention has the advantage of allowing embedding of watermarks 
in multiple copies of the same media signal with small time delays. The storage requirements 
IS are furthermore smaller than if several parallel watermarks are stored. 

The general idea behind the invention is thus to split the watemiarking process 
into two parts, one which is based on signal dependent pr o perties and one which is based on 
watemiark specific properties. The signal dependent properties are detemiined in advance, 
while the watermark specific properties are determined at the time of deliveiy of the media 
20 signal 

These and other aspects of the invention will be apparent firom, and elucidated 
witii refermce to, tiie embodiments described hereinafter. 



25 The present invention will now be explained in more detail in relation to the 

enclosed drawings, by way of example, where 

Fig. 1 shows a block schematic of a device for simplifying embedding of 
watermarks in copies of a media signal according to a first embodiment of the invention. 
Fig. 2 shows a flow chart of the method of simplifying embedding of 
30 watermarks in a media signal the device in Fig. 1 is working according to. 

Fig. 3 shows ablock schematic of a watermarking unit that can be used in the 
device in Fig. 1, 

Fig. 4 shows a block schematic of a device for sinoplifying embedding of 
watermarks in copies of a media signal according to a second embodiment of the invention. 
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Fig. S shows a block schematic of a server device for simplifying embedding 
of watermaiks in copies of a media signal together with a corresponding client device 
according to a tiurd embodiment of the invention. 

Fig. 6 shows a flow chart of a metiiod of simplifying embedding of 
watemiarks in a media signal, that tiie server device in Fig. S is working according to. 

Fig. 7 shows a flow chart of a method of simplifying embedding of 
watermadcs in a media signal, that tiie client device in Fig. 5 is working according to, and 

Fig. 8 shows a block schematic of a server device for simplifying embedding 
of watermaiks in cqpies of a media signal togetiier witii a corresponding client device 
according to a fonrtii embodiment of the invention. 

The present invention relates to the field of providing watermaiks in media 
signals and is especially arrange for provision of watermarks in electronic media signal 
delivery sfystems. 

Fig. 1 shows a block schematic of a device for simplifying embedding of 
watermaiks in copies of a media signal according to a first embodiment of the invention. The 
device includes a server unit 1 0 and a media signal delivery unit 20, which are connected to 
each other. Thefimctioningof the device will now also be described in relation to Fig. 2, 
which shows a flow chart of tiie method tiie device is working according to. The server unit 
includes a media store MS 1 2 including a number of media signals, such as for instance a 
number of audio signals, like a number of songs. The media store 12 is connected to an 
analyzing unit 14, which determine watermaddng properties p that are based on the media 
signal X to be watermarked, step 30. After having determined these properties the analyzing 
unit 14 stores these properties p in a property store PS 16, step 32. This determination is 
perfoimed for all signals in the store 12 and is furtiiermore performed off-line, i.e. in advance 
of any delivery or request for delivery of a media signal to a client or recipient 

When the device receives requests for certain media content^ the server unit 
fixrwaids the media signal x fix>m the signal store and the properties p corresponding to the 
signal to a number of watermarking units 22, 24, 26 in the signal delivery unit 20, step 34. In 
Fig. 1, there are three such units shown, each embedding aunique watermark Wa, Wb, Wc in. 
the media signal x, step 36. The watermarks Wa, wi^, Wc can here be seen as watermark 
dependent properties. When performing this embedding the watermarking units use the signal 
dependent properties p. How this can be done will shortiy be described in more, detail. 
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Thereafter the watermarked signals Xa, Xb, Xo are provided to a sending unit 28, which sends 
the thus watermarked media signals Xa, Xb, Xo to the recipients, step 38. This sending is in 
Fig. 1 indicated with a thick arrow. The actual watermarking is here provided on-lme, i.e. 
when the signal is to be delivered, so that watermarking is taking place just before a signal is 

S to be delivered, which enables delivery on demand even if many clients wish to receive a 
media signal. The sending is of course also provided on-line. The sending unit is a normal 
interfiu^e adapted to be used for the transmission medium in question, which in flie jiresent 
exanq>le is tiie Internet 

With this device a number of advantages are obtained. It is possible to embed 

10 watermarks in multiple copies of ihe same media signal with small time delays, because the 
signal dependent properties have been determined beforehand The storage requirements are 
furthermore smaller than if several parallel watermarks are stored, since only one copy of 
these signal properties are stored, which are then used for all watermarks embedded in a copy 
of the media signal. 

15 There are some variations of this first embodiment that are obvious. First of all 

it is possible to have only one watermarking unit instead of tiiree, which could provide all the 
watermarking. Then it would be having different watemoiarks as input information. It should 
therefore also be realized that more or fewer watermarking units can be used in parallel. 
Another obvious modification is that the units 10 and 20 can be combined into one single 

20 unit They can also be provided in a network; which would then preferably be a separate 
network from the network on which the signal is sent to the client, although this is no 
requirement. It is also possible to provide several units 20, each communicating with the 
server 10. 

How an actual watermarking unit can be made to work will now be described 
25 with reference being made to Fig, 3, which shows a block schematic of an envelope 

modulation watermarking unit 22, The device is a device for watermarking of signal samples, 
like for instance of audio signal samples like PCM samples. This is however just one 
example of the type of signals in which watermarking according to the invention can be 
performed It should be realized that the watermarking principles of the furesent invention can 
30 just as well be applied on compressed audio signals as well as on image or video signals, 
either conoqpressed or decompressed. The watamarking unit 22 includes a bandpass filter 44, 
which filters the media signal x[n] and provides the filtoied signal Xb[n] to a multiplying unit 
42, which also receives a watermark Wa[n] and multiplies the watermark Wa[n] with the 
filtered media signal Xb[ii]. The ou^ut of the multiplying unit 42 is connected to a scaling 
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unit 46, which scales the ou^ut signal from fhe multiplying unit 42 with a scaling parameter 
a and provides it to an adding unit 48, which also receives fhe media signal x[n]. The output 
of flie adding unit 48 is tiien the watermaiked media signal x/Jn]. The scaling ftctor a is 
controlled hy a signal p[n], which signal is made up of the watemiarking properties 

5 depending on flie media signal. These properties are hi tiiis exaxaple decided based on a 
psycho-acoustic model of the human hearing system in order to ensure that the watermark is 
not perceptible to a user or client or provided beneath a masking threshold of die signal x[n]. 
These properties, which are highly dependent on the media signal x, are therefore calculated 
beforehand and stored in tiie properties store in Fig. 1. The calculation of these properties is 

10 quite complex and time demanding, but is however only needed to be made once for each 
signal. Also tiie banci^ass filtered signal Xb[n] is fixed in the sense that it is not influenced by 
the watermark and can just as well be calculated beforehand and stored in tiie properties 
store. This will make the watermarking unit simpler in stracture but also require additional 
storage space in the server. It should be realized that the watermarking properties dependent 

IS on the media signal are not lintiited to being based on a psycho-acoustic mode^ 

hearing system. La case the media signal is a still imagp or a video signal an ap pro pri a t e 
psycho-visual model of the human visual system is used. The model is tiierefore a model of a 
human sensing system. More detail about the specific watermarkmg technique shown in 
Fig. 3 can be found in the document, '^A temporal domain audio watermarking technique", by 

20 Aweke Negash Lemma, Javier Aprea, Werner Oomen and Leon van de Keikhof, IEEE 
Transactions on Signal Processing, April 2003, Vol. 51, page 1088-1097, which is herein 
incorporated by reference. 

According to a second embodiment of the invention, it is possible to also mix 
together two watermarks in order to increase fhe payload capacity of the watermark. A device 

25 for simplifying embedding of watermarks in copies of a media signal according to this 

second embodiment of the invention is shown in a block schematic in Fig. 4. Here the server 
unit 1 0 is identical with the server unit in Fig. 1 . The notable difference here is that the signal 
delivery unit 20 includes two watermarking units 22 and 24 instead of three as well as a 
mixing unit 50 provided between the sending unit 28 and the watermarking units 22, 24. The 

30 serverunit 10 here works in die same way as the server unit in Fig. 1 and will therefore not 
be described in more detail. Also the watermarking units 22 and 24 wcnrk in the same way as 
has been described in relation to Fig. 1 , and will therefore not be described in more detail. 
The nodxer 50 receives the two watermarked copies Xa andxb of the signal x and mixes them 
so that different sections of the signal x is watermarked with one of the watermarks in a 
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sequence determined by the mixer. In Fig. 4 only four sections are indicated in die output 
signals fix)m the mixer. It should however be understood that more or fewer sections could be 
provided The mixer thus outputs different combinations of the watermarks Wa and Wb, 
embedded in copies of tiie media signal, where the number of combinations in tiie given 
5 example can be sixteen. In the figure only three are however shown XA^/ija, ^Afifi^ xb>^. 
It should also be understood tiiat if more sections are used the number of possible 
combinations are increased Another possible variation is to mix more than two watermarked 
signals. The thus mixed signals are tiien provided to tiie sending unit 28 for sending to 
different clients. How this mixing can be performed in more detail is described in tiie 
10 document ^Real-time Concepts for Block-based Watermarking Schemes" by Michael Arnold 
and Oliver Lobisch, WEDELMUSIC Conference, 2002, Darmstadt, Germany, pp. 156 - 160, 
which is herein incorporated by reference. 

With the device in Fig. 4, the payload capacity is increased. The device 
furthermore allows the provision of unique watermarks in several copies of a media signal 
IS based on a limited number of original watermarks with only limited additional processing 
and thus a slightiy slower watermarking speed 

The first and second embodiment were described in relation to a system of 
delivering media signals to clients, where the system embeds watermarks before supply to a 
client This system is a consumer system, where the client is not a trusted party. The present 
20 invention is however also applicable in systems where the client is a trusted type of client, 
which can for instance be a company. This system can for exanxple be used f<»: delivering 
video signals carrying movie information to a cinema. 

A third embodiment of the present invention is directed towards this type of 
environment A server for simplifying embedding of watermarks in copies of a media signal 
25 together with a client device performing the watennark embedding according to this third 
embodiment of the invention is ^own in a block schematic in Fig. 5. The functioning of this 
system will now also be described in relation also to Fig. 6 and 7, which show flow charts of 
the method the server and the client devices are working according to. 

As mentioned above the server unit includes a media store MS 12 including a 
30 number of media signals. The media store 12 is connectedto the analyzing unit 14, which 
determines watermarking properties p that are based on the media signal x to be 
watermadced, step 58. After having determined these properties the analyzing urdt 14 stores 
these properties p in the property store PS 16, step 60. This determination is performed for all 
signals in the store 12 and is forthermore performed off-line, i.e. in advance of any delivery 
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of the media signal to the client The server unit 10 also includes a sending unit 52 which 
sends the media signal x as well as the watemiarfc properties p that depend on the media 
signal, step 62. This sending is in Fig. S indicated with a tiiick arrow. The injformation is 
ftirttiemiore preferably sent using encryption. 

5 The client device S4 includes a receiving unit 56, where the properties and the 

media signal are received, step 64. The client device also includes a watermarking unit 22. 
The receiving unit 56 therefore provides the media si gnal x and tiie properties p to diis 
watermarkmg unit, step 66. The watennarldng unit thereafter embeds a unique watermark 
using the signal dependent properties p in the media signal, step 68. The media signal can 

10 then be used, for display to an audience in a movie theatre if it is a video signal. Dififerent 
clients can then embed their own unique watermark in the media signals fhey receive. The 
watermaiking unit can furthermore be arranged to embed a watermark that is dependent on 
the date and time, such that the watermark of a copy of the media signal that is displayed 
changes from day to day and the time of day. fii this way it is easier to trade in copying 

15 and distribution. 

la tins way sunple and trusted watermaiking units can be placed at trusted 
clients, which then watermark the media signal locally based on the properties that have been 
determined befordband. A content owner can furthermore insfpect the effect of the watermark 
on the content and then manually modify specific features p such tiiat the watermark energy 

20 is weaker or strcmger in di:%rent frames of the signal. In this way he can ensure that tiie 
watemiark is not perceptible. 

The provision of the properties together wifli the media signal extends the 
bandwidth required for transmitting tiie information to tiie client A fourth embodiment of the 
invention is directed towards solving this problem so that no extra bandwidth is needed for 

25 the properties. 

A system according to this fourth embodiment of the invention is shown in a 
block schematic in Fig. 8. Fig 8 is similar to Fig. 5 and only the differences will be described 
in detail here. The server includes a lossless encoding unit LE 72 which receives the media 
signal and the properties p and losslessly codes the properties in the media signal x for 
30 providing a modified signal x'. The modified signal x' is thai supplied to the sending unit 52 
for sending to the clirat The client device therefore also includes alossless decoding unit 74, 
which extracts the properties p without losses from the media signal and then applies, these 
properties in the watermarkmg unit 22. How this lossless embedding can take place is 
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described in more detail in European Patent Application No. 03100093.8, which is herein 
incoiporated by reference. 

With this fourth embodiment there is no need for extra bandwidth for sending 

the properties. 

The present invention has many advantages apart fiom the ones already 
described. The watermark embedding accordmg to flie invention is particularly well suited 
for forensic trackmg, where watermarks are embedded in files distributed via an Electronic 
Content Delivery System, and used to track for instance illegal copied content on the Bitemet. 
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CLAIMS: 



1 . Method of sinq[>lifymg embedding of watermarks in different copies of a 
media signal comprising the steps of: 

determining watermarking properties (p) dependent on a media signal (x) (step 

30; 58), and 

5 - storing the signal dependrat properties (step 32; 60), such that the signal 

dependent properties can be used when embedding unique watermadcs in difiiarent copies of 
the media signal. 

2. Method aoccmiing to claim l,furthercoinprisingthe step of sending the media 
1 0 signal together with information at least based on the signal depending properties to at least 

one recipient^ (step 38; 62). 

3. Method according to claim 2, furdier comprising the step of embedding unique 
watermaiics (wa9 wb, Wc) in different copies of the media signal using the st^ 

IS dependent properties (step 36) and wherein the step of sending comprises sending a copy of 
the media signal to each recipient with an embedded unique watermark Qk^ xb, xc), (step 3 8). 

4. Method according to claim 3 , fiirthar conxprising the step of mixing 
watermarks for providing a unique mix of the watermarks in copies of the media signal, 

20 

5. Method according to claim 2, wherein the step of sending comprises sending 
the media signal together with the signal dependent properties (step 62), for enabling 
embedding of a watermark by a recipient 

25 6. Method according to claim 5, further comprising the step of losslessly 

encoding the signal dependent properties in the media signal. 



7. Method according to claim 1, wherein the signal dependent properties are 

based on a perceptual model of a human sensing systenx 
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8. Method according to claim 1, wherein the 8tq>s of determining and storing are 

performed off-line and the step of sending is performed on-line. 

5 9. Method ofembedding a watermark in a media signal (x)conq;nu 

of: 

receivmg a media signal together wifli certain watermaildng properties (p) 
dependent on the media signal, (step 64), and 

embedding a watermark (wa) based m tiie signal dependent properties (p) ina 
10 copy of fhe media signal (x), (step 68). 

10. Method according to claim 10, wherein fhe signal properties are losslessly 
encoded (LE) in the media signal (x') and furflier comprising the step of losslessly decoding 
OLD) the signal properties from the media signal. 

15 

11. Device for simplifying the embedding of watermarks in diCforent copies of a 
media signal comprising a server unit (10) including: 

a properties determining unit (14) for determining signal dependent 
watermarking properties (p) of a media signal (x), and 
20 - a signal propertira store (1 6) for storing fhe signal dependent properties, such 

that fhe signal dependent properties can be used for embedding unique watermarks in 
different copies of fhe media signal. 



12. Device according to claim 1 1, further comprising a sending unit (28; 52) 

25 arranged to send the media signal together with information at least based on the signal 
depending properties to at least one recipient 



13. Device according to claim 12, further comprising at least one watermarking 

unit (22, 24, 26) for enibedding unique watermarks (wa, wb, Wc) in different copies of fhe 
30 media signal using fhe stored signal dependent properties for enabling fhe sending of a 
uniquely watermarked media signal (xa, xb, xc) to each recipient. 
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14. Device according to claim 1 3, wherein fhe sending unit (28) fUrther comprises 
a mixing unit (SO) arranged to mix watermarks such that tiie unique watermark sent to a 
recipient is a unique mix of tiie g^erated watermarks. 

1 5. Device according to claim 12, wherein tiie sending unit (52) is arranged to 
send tiie media signal (x) together widi the signal dependent properties (p\ for enabling 
embedding of a watermark by a recipient 

16. Device according to claim IS, further comprising a lossless encoding unit (72) 
for losslessly encoding fhe signal dependent properties in the media signaL 

17. Device according to claim 1 1 , wherein the properties determining unit is 
arranged to detemiine the signal dependent properties based on a perceptual model of a 
human sensoiy system. 

18. Device (54) for embedding a wabemiark in a media signal comprising: 
a receiving unit (56) for recdving a media signal together with certain 

watermarking properties (p) dependent on the media signal (x), and 

a watermarking unit (22) arranged to embed a watermark (wa) based oathe 
signal dependent properties (p) in a copy of the media signal. 

19. Device according to claim 1 8, wherein ihe signal properties axe losslessly 
encoded in the media signal and further comprising a lossless decoding unit (74) for 
losslessly decoding the signal properties from the media signaL 

20. Signal for providing media content to a recipient comprising a media signal 
(x) together with certain watermarking properties (p\ which are dependrat on the noedia 
signal (x). 

21. Signal according to claim 22, wherein the properties are losslessly embedded 
in the media signal. 
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ABSTRACT: 



The present invention relates to methods, devices and a signal related to 
simplifying the embedding of watermarks in different copies of a media signal (x). The 
watermaridng process is split into two parts, one that is based on signal dependent p r op ert i es 
and one that is based on watermark specific properties. The signal d^endent properties are 

5 determined in advance, while the watermark specific properties are determined at the time of 
delivery of the media signal. The signal dependent watermarking properties (p) of a media 
signal (x) are determined in a prop^es determining unit (14) and stored in a signal 
prioperties store (16), such that the signal dependent properties can be used for embedding 
moro than one watermark (wa, wb, Wc) in different copies of the media signal. In this way it 

10 is possible to embed watermarks in multiple copies of the same media signal with small time 
delays. 
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PROVIDE PROPERTIES P AND SIGNAL x TO 
WATERMARKING UNIT 
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EMBED UNIQUE WATERMARK w a IN A COPY OF THE MEDIA 
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DEPENDENT PROPERTIES p 
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